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(54) Apparatus and method for summarizlns video Information, and processing program for 
summarizifts video Information 



(57) A summary reproducing apparatus (100) in- 
cludes a detection unit (103) for detecting siient and 
r^olse sections based on information on sound wave- 
forms of Inputted audio^'vldeo infofmation, and a control 
unit (107) for deciding digest sespTsentB to fas extracted 
while contfoliing a reproduction unit (1 06) based on llie 
digest segments. Tiie control unit (107) sets the digest 
segments andtiie importance of each of tiie digest seg- 



ments based on the time-base position and'or section 
lengti) of eacfi of the silent and noise sections in the au- 
dio/video information. Based on the set importance of 
each of the digest s^ments, the controi unit (107) then 
controls the reproduction unit 1 0S to play a digest of the 
audio/Video intomriation. 
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Descrffstion 

[0001] Trte present inveoltOR reiaies to ihs technicai tieid sf apparatuses for reproducing and piaymg a summary of 
video information So which sound is added. Mote parlicuiariy, it relates to the fieid of lecrinoiogy for setecfian of partial 
video information to be extracted at the lime of summa^ reproduction on the basis of the sourid ievei. 
[0002] As recording apparatuses such as VTRs (Video Tape Recorder (VCR)) for recording and reproducing video 
information !ii<e a telsvision braadcasting program have recently become widespread, digest reproduction (summary 
reproduction) has been in practtcai use. The sumnriary reproduaion provides a tjuick sight of video information sum- 
marized in snort time to eiiminate the need to view ali the recorded video information. 

[0003J Methods tor perfomiing summary reproduction include, for example, a summany reprodvicing method In vvi^lch 
scene changea parts (scens changes} are detecled isy focusing mainly on the video In'ormation itsefT: find a rnemod 
for perfoffning summai-y reproduction by focusing on audio jnfonnatior! added to the video information. A typical ex- 
Htripie Cil \he ni&hoti tor performing summary reproduction by focusing or; the audio information ;s disclosed in Japaneso 
Laid-open PatsfJt Appfecaiion No. Hei 10-32776. 

[0004] As srwwn in FiG S, a SL-vnriV'<y '^cvoii^ichy ;n':psr?.t>;s i disoiosed -n tins Japansjse l.aid-Opfsn Pa-m Appli- 
cation includes thaJoiiowin'i^' s f.ound «:■..-■;:■::-':;; ;r s:.-.rs 2 f3rdf;;ec-;inglheso;.ind ievei of video informaiion providi^d 
ox'cr 8 comrnunicBtion iins; or aiiWove;. i;:.-.K;-.:;s.>! -v I? ;:jd;o ;n?o'm.?,l;on addod to ti^c- video information (hefeinafier 
called auditt/video i.ilofnaalion): a comp.?rc:tc" ;or :,;.;r;j.^r:ni;; ihe sc\jr6 yvc-i vv:!'; ^\ refusrence; souad ie-vei: a dLifaiion 
titror 6 fo' measuring ihs diitation of finie dunng wtiicfi iNe soli^-c; :i.'v:; occi-'us i le -e-efef-ce sound ievet: a digest 
address generating means 8 tor generatinq addresses cf c^ges; p;;-;;; U o^r .no c^'-dion .measured by "me duraitori iime' 
&; a mcording.'Vapfoducincj means 9 h- recording ihe acirj-i.-Ksof.- c::;;;,t;: <)o^rr:~:^ reproducing means 11 for repro- 
ducing the addresses recorded: and a repiay control means 1 0 for playing the digest parts of the audioA/ideo infomiation 
on the oasis of the addresses. 

[0005] According to the above-mentioned configuration, men the input-ed auaic/video infomiation iasts for a preset 
period of time during wnich the sound ieve! of the audio/video information exceeds the refe;-ence sound tevei, the 
summary reproducing apparatus 1 records the addresses at wh;cn the sound iex'o! tjscomes nigner than the reference 
sound leviji. T nen. the SLiirtmasy reproducing apparaius 1 extracts, based on ihe addressoS; the par;s the sound level 
of which becomes higher than the refsrence sound level to reproduce a summary of the audio/vidao information from 
the extracted parts. 

[0006] However, \d the above-mentioned summary reprodiscing method, onty the parts the sound levei of which 
becomes h^her tiian the reference sound ieve! are used as feature parts of the audio/video inJormation without the 
lis© of silent parts of She audio/video inforniation as Its feature parts. This causes a problem of being incapabfe of 
performlrjg proper summary reproduction. 

[0007] Afl audio pail the sound level ot vrfiich is high (hereinafter called a noise section) indicates an exdting part, 
and hence a feature part of the video Infomiation, On the other hand, a soundless, silent part (hereinafter called a 
silent section) indicates a part that changes scene or switches the contents. From this point of viewf, it can be said that 
the siient section is also an Important feature part of the video information. When the contents are switched in the video 
information, the immediately following part is the beginning part of the next contents and often gives a short summary 
or outline of the contents concerned. 

^0008] Thus, the above-mentioned summaty reproducing method can extract exciting scenes, but not a(! the scene 
change parts or the parts that switch the contents, resulting in the proDiem of tsetng incapabis ot perroCTing proper 
summary reproduction . 

[0009] Further, since the above-mentioned summary reproducing method is to play, at the time of digest viewing, sll 
the parts of the audio/video infomiation that have sound levels higher than the naference sound level, it has another 
problem that the audioifvfdeo information may not be summarized In a piaying time re<}uired by a user or preset playing 

time. 

[0010] T IS p c=e 1 ny -^r ton hd=i b( i - ntobicns and ft is an object thereof to 

o'Cidedi o !! to mc- ion ox ifClec ^ i*f t t nose parts so that an oporato' 

dn \> T] om^^ int r -1 c d'-'j. v v, 

[0011] Tq<,aLiv^oLf- n ^ ^ - n wtztrg apparatus of 

»Koo,-,c;(,qt (,rt! nfc M ■= 0' Video in'otmation 

fa^cdc-^audv^ -fi^rnat t o ^ ■> -vi, hatdf e mon-nat'^n 

ur 1 ' 'cf^ t =; '^rt'^! net^ ^ "incntncb n oUnc 

lo! V to rti r " t ^ >. *c da's tfying v oeo 

r^f I -icit! fi nt pkiMl yir Lctrn-^cn ht. bas of ihe ourdievoi tn h^.-ucii. f->f i! ri s Jvi'- n devict fo 
oecidmgthepait 1 do i> ulcnto jjextfacttX!ontht>>-d'it&of atio^is,tether hs. l r ^ 3 o <. iir no the time 
!org*h of at e^si rnv cn^ r i'--^ piurat typss sound sectms c asst^ied m the v'aeo inr nv or and r qpneration 
Device for extr^c tmq tne iei ide d ^a tiatvraeo n*o'mat(on*'-omthevdeoi'ifcnTi >ionlog' t rite h-^ > ^i. inf'jrmd ion 
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[0012] According to th© present invention, the ciasstScation device ciassifies the vWeo information into pfurai sound 
sections on the basis of the sound ievefs in the audio information, tii& decision device decides the partiai video InSor- 
matton to be extracted on the basis of at least eitfierthe time-base position or the tfnie length o? at ieast any orte of 
th© plurai types of sound sections ciassified in ttie video infonriation, and the generation device generates the digest 
Information summaTOed in shorter time than the video information on the basis of the partial video infon^ation. 
[00t3j In generai; since the audio infonnation added to the video information shows feature parts sucti as exciting 
parts of a program, scene change parts, and parts that switch program contents, it pfays an important rote in summa- 
fmng the video infofmation in shorter time, 

[001 4J Therefore, since the partial video information to be extracted can be decided on the basis of the plurai sound 
sections ciHss Ned by sound leveL both the exciting parts and the parts that switch program contents can be extracted 

e. s ihe odfuai vdeo Information thereby obtaining digest ntormation tnat enables the user to grasp the contents un- 
erringly in shon time, 

[0016] ir one asccct of inc. afcs.sni tnventior, tho decs:^- anwice doc dos at least ejtherthe start lime or the atop 
tin^e of tl-iO p^rtwl videjo [nfc-rr-^'ion or- the L<c)sis of at ie-Hbi c rte! f;o Itn-^-b^tse position or the time length of at least 
j-n*/ ono ;)f ^nr- rk '^i^ ty. O'^ of t^'-m-.ci wUor-i (.•i;isst''Od n the video (nfo'matson 

[0016] AccotdiJicjto "nti.<;5pe;i ire doc-!ft::n dpvco decides al least eitherthe start time orthe stop time of the partial 
vioooinfo-nidtior ;n ti e br.sts c i .it ifiaft o t!-or!hi> time-base position orthetimeiengthof at least any One of the plural 
types af sound sections ciasstfied in the video fnform;ition , 

[0017] Tne,r«,;orc feirce ttic pftid types of 5-.ourid iecUOFis c cisst'icd by s>ouf-d eve show excS og Cd'ts y the v:cic'0 

mfonnation scene charge pans, and paits ;hf;l ?wilc^ contents I'^c'sc fcti^urc ptjrts can &e ocfacted as f^e parfial 

video infofmation linernrgy on me b<jsis of tne pus'- types o' soi.nd sec- nns c^assif sg hy sojnd ievel iheisby ob- 

taning appsopnats Cigest irJcrtna^ion th^t enables Ine -.sei tc cj'-asp the cor-tenfs unstnngH m shoii-l-iie 

[0018] Ir another aspect of the p-esent invention the class.? :a.uon device ciassnes 0"i >he t^ssis oi t^'e souna k-vos 

tie Video nformation nto at e^jst sou-'d ess s^ent seotics aid f-o'se secbcns thai taH wrtnm a pieset range of sound 

\e\)(i\&. 

[001 9j According to tf-^is, ,.b.poa the- clAssrccinor ciev cc c -.is ' c* 'he b?s s of tf-^o sctjrjd i:A-ois ihe video nfor- 
'Ttation rto at least soundless stiurf sections nr-o r-osse acclionb thai 'till within a ptese" irfnpo of soijna l&vets 
[0020] In general bott' tno silant and nosso sections play important roles in sumnanzsng the video infonrtation in 
shorter time. For example, m a television broadcasting progr:im, a noise section higher -a sound level than a preset 
level indicates an ex&ting part of fhs ptogram, while a silent section pteset sn ievef as being soundlt^s indicates a 
scene change or a part that switches pr<^ram contents, 

[0021] Therefore, since the partial video information to be extracted can be decided on the basis of either the silent 
section orth© noise section, both the exciting part of th© video infonrtation and the part that swSches program contents 
can be extracted as the paniai video information, (hereby oiataining summatized video information that enables the 
user to grasp the contents unerringiy in short time, 

[0022] In further aspect of the present invention, the decision device sets the start time ol the partiai video Infonnatlon 
at a tsme-bass positron that shows the end of a corresponding silent section havtna a preset time (snath, 
[0023] According to this aspect, the decision device sets the start time of the segment at a troe-base position that 
shows the ©rta of a con-espondinq silent section having a preset time tencgh. 

[0024] r «^\,do nfo '-^af or to v ch f^e a dio nforn on is added s nee the oundiess siiert ection r c te 

f. cer^i. aiqcp t apal hat^v s f^n r tltp t tt at immedtatsly ollew the s lent ecttot becomes tie 
Svt g t t 4 o t Ft t } q part '-ften q ves a hort ummari oj outi ne of t ie 

contenis n becomes a feature par; ot lae viaeo infomiaiion 

[002 ] T the en po<?!t on of tne lent 

«8ct!or \ -i w-i io ) f c can be extracted tnerr nqly 

[0026] f pr>c 0 1 n d! V deo n^ofmation faa«ed n 

nc= r n 1^^ de 0 d c ba edorl etmob r- t n 

0 e ret tt^ t 

[002 j A g 1 t. n ba«e n t e s t o 



[00281 t c- a ca t i 

nr ato v aftc i it c c v>« 

r atoy -5 ie { " 3 }. . r{ tn 

tinie axis inmediateiy loHc-ws ti-io ouliine p;srt as a ieaiufe pan, and it is a good pit:ce to le«:VG off. indicalif-g such prope;- 
timing that the user wii! not feel something wrong at aii even if the part is cut. 

[0029] Therefore, since the stop time can be set on the basis of the silent secticn tiist foilows !he start time of the 
partial video information, the partial video information can be extracted as sucti proper timing that the user can view 
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the outiine of the feature patt without a feeiing o! wrongness because the silent section is a good place to leave off. 

Tfuis. digest intofmatiQn capable of teing the user the video fnfonriatson accuratefy can be obtained. 

[0030] trs further aspect of tiie presem invemion, the decision device sets the start time o! the partiat video Snformation 

based on the timfs-base position that shows the start of a noise section having a preset time iengih. 

[0031] According to this aspect, the decision device sets the start time of the partis! video ifjfonrjation based or^ the 

lime-bass position that shows the start of a noise sectior^ having a preset time length. 

[0032] trs trie video information, the noise section is art exciting part, Jhat is, a feature part of the video information, 
and sspeciaily the start position of the noise section piays an important role in grasping the contents. 
[0033J Therefore, since the start time of the partial video information can be set at the start position of the noise 
section, the omv-il video irfo-ma-ion that forms a feature part of the video inforTiation can be extracted unerhngly. 
[0034) ir iuither aspect of the presem invention, after deciding the start time o? the partiaf vtceo inronriaiion based 
on the r;0:se section, the decision device se-s the stop time of the partiai video information based cn the tiine length 
cA the noise section concerned. 

[0035] According to ihls aspect. Jiftt-r deciding the start t;rne of tiie p;-;f!!af vid^X) tt;}o'niat:on baaed on tite noise 
section, the decision device sets the stop tirne of the partial video information based on the tinis length of the noise 
section concerned, 

[0036] 'i'heraiore. s-.nca -he end po,=>ilion of the exci-ici qa-- o; |;.:ri :>^.v. j-.d^so intormaiion can be set un- 

erringly for the partiai video if!fQ!m.ation. the partial video Intosniation caf: be axtracted at such proper timing that the 
user vviil not feet something is wrong at ail, iherebv obtaining digest information capabie of telling the user the video 
snfomiation accuratety 

[00$7] In fijfther aspect of the present inventson, the decision device sets, within k preset time range, the time length 
of the partiai video iniormatior: to oe extracted, 

[0038] According to ttiis aspect, the decision dews sets, vi^ithin a preset time range, the time length of the partial 

video information to be extracted. 

[0039] it one piece of p.^iitiai vjdec Information to be ex-raotsd i,^ too short, the user cannot t;nd>?r:;faiid the p-irt of 
the video informalicn. On the other hand, unnecessarily iong iims length could contain a iorof neediess inlcrmatson, 
and an increase in information amojii; maicss it inipossibta to summatii^e the videc info-miation uneiringly. Therefore, 
it is necsssaty to set a proper length for the time length of the partial video infomriation in order to iet the user know 
the contents of the entire video Information prope.'iy from the summarized video information. 
[0040] Therefore, since a time length enotigh for the user to understand the coi^ents ot the oj^raclsd partial video 
information can be secured white preventing the time length of the partiaf video inforniation from becoming unneces- 
sarily iong, digest infon^iation that enables th© user to grasp the video information accurately can be obtained. 
[0041] In further aspect of the present invention, th© decision device sets the importanos of the partiai video Infor- 
mation based on ai least either the type or the time length of the sound section used a« reference to the decision of 
the partial video information to be extracted, and the generation device makes a summary of the video infomnation by 
extracting the partial video infomiaiion on the basis of the set importance of the partial video informafion, 
[0042] According to this asoect, the decision devtcs sets the importance of the partial video infomiation baaed on 
either tne type or the tsme tengtn of the sound section used as reference to the decision of the partial video snf otmatson 
tD 00 extracted and the gonora*ton devce maltes a summary of the video infoTni^on by e>*ractn8 tho partial v dec 
r'oi^-nation 'hp basis o*' 'hr se- 'moo lance o tho partial v doo if^format on 

[0043] i i?i? or>, jtrot t ic v d\i in'o m-?tio ^ ib sumnan?od on .ic bss s o the importance of tie partial vdeo 
t c lit )ii n^6">ti fi-tnidi ^ ncdiji-bc otc i ro5 ou'tvtr ') tc ^ bh -rlof tr <> lot gt i b. i^cifi^dbythe user or cressl short©' 

time length in wnich the video information is to oe summanjied can be obtained, 

[0044] I-' .h ir^wc ofliii'p. , ^ s K iitk Aii Motof ifi< V.' vo n'o 

mation oased on tne stisnt section than tna- o! tne partial viceo iniormation base-a ors the noise section. 
[0045] Acco dinglr>tnisd--| cct m^t "^o - -a - ■• "^a- d o Tn - ^itt-rtt-v v 

tn^n *h=)t of l-ie o■^ i-al v "c i'-!«Ftra'io i bct=-~" - s 

[0046] Af*hoi.g'^ bo h heiosi -iT" ^l^G^ c 13 '■::ise ^octon n-i 

catesrnexetin [,.art of the vi^ou m oin- '••^ > - _ c.^e^ ss,c-c ch nL,c p<^rt ot a ^fct 

bWi'Cies cf^nte'^'i! n fe V Jiio nfvjruatun ^ cc - v^co c mut oi ba^-yd ot th; si 2it s> ^ tot ^ 
inore importance than that of tne noise section , 

[0047] fiir,i'0!n !5in.;on 0 ? Tip..rt-ini.f Ci'" ^- „ ^i- r--rti(^ mib'"^crton "t-iinot-^ebfc* 

c-n cti'~q s njc .ancs n 0 brtlancj With * "tat u ^ <_ ^„ ^ ^ ' Lf rng^'f^ inq J go^ t' o>tM toi 
[0048] ' rtnet c-sps'totthoptcson invent „ „ = cccs ^tOr-'^irtl 'j e^-infornc- nt 'otn.^-^ 

vHi o K a littler tf d v-i. 10 r fev ca me gos t^ „ . ^ _ - ' vQi^o irfotrriatiot nt a ...1 ce 0' pa tiril 

vtceo mfor'tiaiior . nc ,1 i n nipotlf 1 a of t^ o ma'c 0^ r i ft ! id-^c info m^ton tased ,jn tfit- trrport^nco of tach 
piece of partial v d-x: 1 ntrrr-a nr belt q tr-^'-qoc r- ^ 'cson 

[0049] According tc hif a spect nen the decsded plurai pieces of partiai video infonnation coincide with one another, 
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the dectsisn rfevice merges the cQincident pieces oi partial video intoiroatioti into a piece of psrtia! vicJeo infcrmation, 
and sets the importance of the merged partiai video information based on the impoitsnce of each piece of partiai video 
infoimation being aiready merged. 

[0050] Since such a part that one piece of partiiaf video infomiation coincides m\h another ptsce or other pieces of 
partial video (nfofmation is composed of piorai feature parl^, this part can be detemiined lo ise an important feature 
part in the video infotmation. 

[0051 J Therefore, since the ptiiral pieces of partial video infomation that coincide with one atiother can be merged 
IQ sxtrac! a piece ot partiai video information as an important feature part of the video infcrmation, digest infonnation 
cm i3e oDtainea unerringfy. Further, since the importance of the partial video information extracted can be set on the 
basis of sne importance of each of tne plurai oftrtial video information being already merged, appropriate digest vidso 
inJonnation that enables tne user to grasp the coptenls in short time can be obtained. 

[0052] The above object of tne present invention can be achievad by a video infcrmarion sumn-:an?tng method of 
she pres-sni invenho-i for extractirg. based on audio infonriation one or more pieces of partial vltieo information las 
sorns parEs of video infornsitton from tha video informaiion to which Ihfj audio mfcrrna-ion is added so ihat d:c}ost 
infoE^Tiation SEimmari?:ed in shorter tirtie than the video infofmation will bo generated from the video tnfotniatfon on the 
basis of !iie partial video inforaaiion extracisd The melhod is prcvideci with; a classification process of classifying the 
video iii'ornia!:on into plural sound sections on the basis o! the sound leveis in the audio inforriiation: a decision piocess 
G! deciding Ehe partial video intoimationto be extracted on the basis ot at least eiit^erttie timc-ljasc position or the time 
length of at least any one of the piura! types of sound sections classified in the video inforrnation; and a generation 
process of extracting the decided parliat video infomialion from the video infomiation and generate the digest infor- 
mation. 

[0053] According to Ihe present invention, tne ciassi^fcation pmcess is to classify the video information into plural 
sound sections on the sasis o: the sounc levels in the audio irfomiation. the decision process is to decide the partial 
video iriiormvuion to se extracted on the iDasis of a- leas; eit^-e- the tinns-base position or the time length of at least any 
one of the plural types of sound sections classified in the viaeo inrormatfon, and the gensra^fort process is to gsnarate 
Oigest information summarized sn shorter Time than the video information on the t>asis of ihe partial video iiifDrmaison. 
[0054] In general. s:nc3 the audio intonnation added to the video information shows featijie paris such as exciting 
parts of a program, scene change parts, and parts that switch program contents, ft pfays an important role in aumma- 
rising the video infomiation in .shorter time, 

[00S5] Therefore, since the partial video inform^ien to be extracted can be decided on the basfe of the plural sound 
sections classified by sound levei, both the exciting parts and the parts that switch program contents can be extracted 
as the partial video information, thereby obtaining digest information that enables the user to grasp the contents un- 
erringly in short time. 

[0056] in one aspect of the present iftvention, tJie decisioh preces* decides at least either the start time or the stop 
time of the partial video information on the basis of at ieast either the time-base position or the time length of at least 
any one of She piurai types of sound sections classified in the video information. 

[0057] According to this aspect, the decisiofi process is to decide at least either the start time at the stop time of the 
partia! video information cn the basis of at least either the time-base position or the time length of at ieaat any one of 
the piurai types of sot^nd sections dasstfied in the video information. 

[0058J Therefore, since the plural types of sound sections classified by sound level show exciting parts of the video 
infoRTiation,, scene change parts, arid parts that switch contents., these feature parts can i>e extracted as the partial 
vidso infon-nation unerfingiy on iiie basis of the piurai lypes of sound sectsons classified by sound level, thsretjy ob- 
taining appropriate digest irifomnatlon thai snabies the oserto grasp the contents unerringly in short time. 
[0059] in another aspect of the present invention, the classification process classifies on the basis of the sound levefs 
the video information into at least soundless, silent sections and tvoise sections that lali within a preset range of sound 
levels , 

[0060] According to ibis aspect the dfissi? c-:i o - r !;;-:<. ;i; .s ■■ on the basis of -he sound ievets the video 
In'onriatlon into at least soundless, silent sgc:icii;; .r-..- i;;:.-. ; \rA 'e.,'- Within a preset range of sound teveis, 

[0061] In general, botf^ the siisnt H'^d noise s;c:i:.r; ■;i--iy n'-sc'-.ant rctes m summan^ing the vidso infonnation in 
shorter time. For osampls. in a Miivisicii o oadccs.'. " i .-l oerii :-, a noise section higher in sound level than a preset 
sound ievei indicates an o>«citin^ o?.a ri-i-; r-fijai-i =; iiiont section preset in isvel as being soundless indicates 
a scene change or a part th.%t switches progi-am contents. 

[00S2] Therefore since the psni-^i vicieo ii-^cn-i.^ric " i.c ee ex-'acci ca '■. 2e riacidec; on the- basis ot either the silent 
section or the notse sectior, ijcn ihe exrii -i;; d hi' rf ii'-e ;:2 :•■ i -tc-i i7~-,'. c A'-.6 the part that switches program contents 
can be ortr acted as parti.^i vi-iieo infor-,Tiation, thereay ooiaEning sijmmai!:?ed video itisom-iatton that enables the itsei 
to grasp tt)e contents unor? ingiy in short tin)o. 

[0063] In further aspect of the present Invsrrtion, the decision process sets the impottance of the partiai video infor- 
mation based on at ieast either the type or the time length of the sound section used as reference to the decision of 
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the partial video information to be extracted, and the generation procsss maSces a sijmmaiy of the vidso infofTnation 
by extfacting ttie partial video mformation on the basis o{ the set importance of the partial video information. 
[0064] According to this aspect, the decision process is to set the importance of the partia) video infom-saiion based 
on at feast either the lype or the time length of the sound sectiori used as reference to ihe decision of the fsartiai video 
information to be extracted, and the generation process is to make a summary of the video information by extracting 
1h© partia! video information on ihe basis of !he set importance of the partial video infomiation. 
[OOSSJ Therefore, since the video information can be summarized on the basis of flie importance of the pattiai video 
information, digest information capable of corresponding to a shorter time fength specified by the user or preset shorter 
iinne iengih in wnich the video information is lo be summarized can be obtained. 

[00S6J Tiie above object cf the present invention can be achieved isy a video information summarl/rsng program of 
i?.e present Ins'enilcn embosisd in a recording medium which cati be read by a computer in a video Infofmation sum- 
marising apparatus for extractinQ; based on aiiciio information, one or more pieces of pattiai video intonnation as some 
priHs c; vidfjo irt'crm^tlon from W-.s video informsticn tc whic-: :hs audio ;n!onTiatiC'n is added so that d;c]3st Information 
SEimmarizcd In shcrtartimeti'san the video information will ba generated from ths video information on the basis of ths 
partia! video information extracted Tlie program causes tt'^e computer to function as' a classffication device for clas- 
sifying tiie video iniormation into piurat soiind sectionf: or tl-e bas:s. d I ;-;; .-ro iovols in tl^e audio Inforrnaiion; a 
decision device; for decidirfg the partial video infnrma;icn tc ba ex; ra :. .;.>■..: ■■; ; ;;f:sis. of at least eillier iiie time-base 
position Of tl-ie time iength of a- least ;jriy ot^o of t.ne ■\'pes o- o-;s classified In tiie video infamiatlori: 

and a generation device for oxtraciing the declaea Daftial video infcfrriaiion from tiie video information to generate the 
digest information. 

[OOSTj According to the present invention, its cor-ouis^ cias-si^o.? r-:-. vioso info;mation inio pkiral sound sections 
on the basis of the sound levels in the audio Informat^or, a&iiaes t^a pa'iiai video mfom-ation -o be exirscted on tne 
basis of at least either the time-Pase position or the time length of at least any one of the piural types of sound sections 
classified in the video infomiation, and generates digest infornation summafl;£ed in shorter time than the video infor- 
rnation ors the basfs of the pasttai video infomiation 

[£J068| In general, since the aadio infomiation added to the video intotrnation shows feature parts sucti as axcsting 
parts of a program, scene change parts, and paits !hai swiich program contents, it piays an important roie in summa- 
rizing the video information in shorter time, 

[0069] Therefore, since the partial video Information to be extracted can be decided on the basis of the plural sound 
sections cfassified by sound level, both the exciting parts and the parts that switch program contents can be extracted 
as the pafliaf video information, thereby obtaining digest information that enabies the user to grasp the contents un- 
erringfy in short time. 

[0070] in on© aspect of the present invention, ths decision device that decides at least either the start time or the 
stop tiine of the paftiai video infomtaiion on the basis of ai least either the lime-base pomon or the time length of at 
least any one of the plural types of sound sections classified in the video information. 

[0071] According to this aspect, the computer decides at least either the start time or the stop time of the partial video 
infomiation on the basis of at least either tne time-base positton or the time iength of at leasr any one of the piural 
types of sound sections classified m the Video infomiation. 

[0072J Tneie-oro smce the plurpi f^'pes o* sojnd sections ciassifled by sound level show ©xatsng parts of the v deo 
r'oi^-nation sc"np charge pars and parts hat switch contents t-^eso foatute parts can .>r exfacted 4S t*'? partial 
Vif^eo notnajon Jieinrgyoi h.-' ba^if of the p umi types f<i tnu =oc on'- '^a^-il .■'u by bO jnd ie cl {fr-^ -^ty 

ig^cf'pf'^^' ^i If fo )T ijoi SI ft 4. lititosiho Jwt to ^<-p iho or i,ni Uieisnjiyi bh tmi, 
[0073] I-- mntt £>- i-pe;;* -f he pros -^t ivo" i^- > c t. or de\ ce h ciat jfo on t-obri=sc tt-o - nJ 

evtic h \ct- nforr^^mnirto^tlPc l-^ojndle nl^ ui and ncf espctons i!i t^ sllv thtn j «f an^-eof 
sound iGveis. 

[0074] Accv" ding ^ tf I - pv^ nev.i.rip ^-.^ < ■^ordl^-vcK h vljC r ^n-jl^nfto 

c' e-3bt =^0 jnaio=s si eit =iGdtf^n=i and not'-'' - mqc f^f io ev.cK 

[0075] Ir ors a boti 1 e siler a Jr-^^" > sLn na i q 1!"^ '^rt,:? r ur—i.^t -^n r 

=h<^i1fc.r ie "O'-cx^mx.le n ter^viston b -'^^ chct r --■^u'^u le^el thtr ^ ^t'•e^f>t 

'-vHdit,','' t d -itoati f='X<,it n Ocirti^ftlippr ^ c a--beit q ^ot.t o e«>s ii J> - e-- 

s^<r^.r^'aqo ! a c^'" 'h t =1 v cics prog - v v, 
[00761 f "1 ^'0 " --inve tti{> part idco niation to be cxttac e'^ -^a i -^o d„ride-^ -^r t le oasts ot e ti ei t) o s lent 

ectyf ■j'^lo m^cs'Krtn'' ijQthtx ^xLttn ; part of the Videoirfot-mationapd the part that switches proQ! am contents 
c-i 1 i:^. Axtf -ictv.d -s J. art - *,rL m c f -nr- ion. thereby obtat tng sjmmarzed v deo tnformatton that enables tf-e user 
50 grasa tt!o cc itotns unei'ingly m short time. 

[0077] Ir further aspect of the present Invention, the decision device that sets the importance of the paittal video 
infomialion based on ar least etther the type or the time iength of the sound section used as reference to the decision 
of the partial video Information to be extracted, and the generation device that maices s summary of the video information 
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by extracting the psrtial video informatton on the basis of the set tniposlance of the partial video information, 
[0078] Accofdiog to this aspect, the computer sets tlie importance of ttie pattiai video infotmation based on at least 
either the type or the time fength of the sourKl section used as ref erence to the (Jecision of tf^ e partJa! video infomiation 
to be extractsd, and makes a summary of the video information by extractirig the partial video information on the basis 
s of the set importance of the partial video information. 

[0079] Therefore, since the video information can be summarized on the basis of *e importance of ttie partial video 
information, digest information capote of corresponding to a sh orter time iength specified by the user or presat shorter 
time isngth in which the video information is to be summarized can be obtained , 

10 in the Drawings; 

[0030] 

~\G ^ J b o(.k <<i gfrim st-ow ng tho structure of a sunrf^ay repfoducmg apparatus accotatng to an e-rtbodnient 

t5 of tne pfssent invention: 

1- iG " \i <iq djh to e>piri! trq how to oct a ^ 'oi -"U t- rf-'d .-t peso sect -ii scco'n;in tc tic "tdc^iti-^i 
HG 3i!=f diigirii'oroxpr-nntj icwtoJe,"tai> '■■''s*-"'"-; fdiioplinx aJ i^rirnc- b-ooc cntho iof>c <pc> o,i 
\(j -i - a<i cifjf-ir ^ tor i^xo i ^i■K! "ow tc 0£. ^ ' i « s op y io <) segment Lc^^jd on .^lic'■t* ^t.f" oi 
"iG )s 1 iiowoi^d'' sti' v,ff g dfgcs. ^t,:; ^ -^tiot '■if s-jrnndiy efro tor ciiv'C'r^lufi 

20 gjTibodiment: 

6 )S f { owchurt shuw It, r-i s(^t irg opa - . o -e^'cjcqf^ss Rygrtpr c?e("(cec!0ith8 bastsof tne 

noise ssclio ir t^^e s trn a v ^p'Oi^uc ic" ^,<^ V ' r-.,cc Qf- o f-^ sTibcdtne it 

7)safo'Achdttshuvv ^9asl.ti^go^3c v. o' v - '^v -^e o d c gcs seg^ier ceciced on the basfs of tie 
Silent section in the summary reoroduclton operatson according to the emDOOiment: 
S5 FiQ. 8 IS a grsaph tor sxpiainins how to oetect pturat notse sections according to the embodirtiem; and 
FiG. 9 ts a block disgram showing the structure of s conventfonaf summary reproducing apparatus. 

[0081] A preferred embodiment of the present invention wiii now be described on the basis of the accompanying 
drawings. 

30 [0082] The embodimem is carried out by applying the present inverttion to a summary reproducing apparatus for 
summarizing and reproducing audio and video infonnatlon such as a television broadcasting program provided over 
a communfcations iine or airwaves. 

[0083] Referring first to FiQS. 1 to 4, the generai structure and operation of the summary r^roducing apparatus 
accordirtgto the embodimertt will be described, 
as [0084] A summary reproducing apparatus 1 00 of the embodiment shown in FIG, 1 takes in digital audio/video infor- 
mation transmtEted from a communications iine or received at s receive unit, not shown. Then the summary reproducing 
apparatus 1 00 decodes the inputted digital audio/video information, and separates audio information from the decoded 
audio/video information to decide or select partial video information (hereinafter calfed digest segments) to be e>d:racted 
for summary reproduction. 

40 [0085] The process to decide d?gas5 segments to be extracted Is carried out. as follows, pfjtentia! digest sfjgmenls 
ii-iereinaftsr caiied digest segment c«ndidafes) .are listed, and then digest segmwls to i)e extfact!5d are narrowed down 
ficjtv! the listed digest siKj'netn aindid.Htes to cieade digest seg;ner:;a io be us'jd for Siumo-iniy tep:-odiiction. 
[0086] Tnis process to decide tho d^gss; S8g'~-:snts :S Cftried otjl Dy obt.-sning time irv'ormEiliori such as the siart and 
slop lini;; cof «acii digest K:-?gn:«;;t ara ;!•■;: . '.he dijwst segment. Tt^en aigesl segrnfjriis yie eytfac.tsvd irom 

^ the inputted digital audio,/video inio-n-:«L.c:-: dus'^l: •.^■^b decided time infofmation and order ot importance of the digest 
segments, and the extracted digest scgrr-^onij --e vcniinuousiy reproduced along the time axis there:nafter called 
summary reproduction:). 

[0087] It should be noted that in the eT;bodimen; \'!deo infonnaiion and audio infonuation added to the video infor- 
mation are multiplexed into the digitas auQiC'Vicso information. 

50 [0088] As shown in F:G 1 .. i-ie sj.-.-?-/ r-g apparatus 100 of the embodiment includes a dsmuttiplexer 

101 for dctriviitiplex^ng mc >.^::dl-:. r'-c-^-.r;--^ 'i-c irtputted digital audio/video Information, and a decoder 102 for 
decoding the .^udio infosi-nation as digirs! signals aemultiplexsdby the demultiplexer 101 to obtain infonttatiort on sound 
waveforms (sanpic v.^iLtes ■f:.2;s:i-f.'t£-c.a s;„:-c. ■,v--ve'rr".- ^"'o^ rv,: ThesumiTiary reproducing apparatus '00 
also Includes a detection un:t 103 for deiocting silent sections and noise sections from tn© sound waveform infcrmation, 

55 -A Storage unit 104 for storing :r:;oririat!on or- the aetected silent and noise sections in tt)e audfo/video infotmation 
concerned, and nr. oparf-- 1 on l; n it 1 05 for use in operating each unit and entering the iertgth of time in which the audio/' 
video Information should oe summarized, Funher, the summary reproducing apparatus 100 Includes a reproduction 
unit 105 for performing .surmiafy feprodudion of the stored audio/video infonnaJion, a confroi unit 107 for deciding 
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digest segtion's tf be ci^l i l:£,f}cm*h^ l l idt^/vi^LO n'otn m ici ruttt ) t.|.t di tnr in 0^ --t 5 
disptay unillOSfo di^p lyin itht-sj nt aij'oacsndte-^roductdviaeo&iqr jswh t-OLttpdltirsjc soi.Mtt.d t s j>i il 
[0089] The detection unrt 103 constttULes a ctassificalsofi device accorcJtnq !o tne preserit invsmton. while the control 
mk 107 and the rep oductron i,n 1 106 constitute a :iecston device e'nd d gcre'atror* oevcc dccordtnq to the present 
invention. 

[0090] The digita! audio/video infomnation sen): from the commuracalions line or receded at the receive unit, not 
shown, or the digital eucfo/video infonrtation tnat has almsdy been stored in the storage mart 1 04 are inputted into tne 
os'ttuit piexer 101 The demutiplexer 10' demullsDtexes t!~e audio rjfornatior froir the nputtsd digsta! audto/vdeo 
mformaiion, anc outputs the demultiplexed audio mfotmalion to the decoding unit 102, 

[0091J TT=cigt lai.doirluf!ralo'icUpjt1eof''on hedeTiuitplexei 101 ^mot ea i othsoecodng jmt102 Tne 
'^t. od p j2 de'^cUos the nputed digit i aud o tntormatoi ootamss uid wt^xef-^m mtormpjton from tie audio 
ff -nH-^ rs ou f.u! tei-ttfiiju uurd Aave^o m irfoTnatoi t(, n os t^tto'- umt 0 

[0092] ] 1 if^wd r-i I o!^iaf>i!-j n„L leo ton <- -"croJ i uni! 0^ non^os'c t]orunii10< 1 if> ap 
* vt ! b 1 " jc. xto'flu t re t n ritLCi.\. f '^m^.u l is i dv^rt^i.-' in on i c i 

[009-?] I the-nb :! riw ! itv\n ^ \ ") j i s ti- 1 t nf^ o i lor hoto t >n r 

=-T"l>i.dl 1 '-■^si po'- I0l^^' „ inJcF blip!/ i\M i^ v^^o iJ-^itc cacho 

c i! f r-i J ir <,sxlrr->r -3b = ti p -^liori -'vr (hfo<=i^.lr c ci-^-t red ■'J 

cvo I sfi I ( J)Knd.i cijOt ncisoljvciii-toshold(hoto(f ate cafiedtno -^Ofao lav^lthioshoid JHf,)i 
Then the aeteaior 503 outputs to Ifie storage unst 104 time mfomialson on the start and end Dosttions detected 
for each oi the silent and noise sections. Hereinafter, the length of time for each of the silent and noise sections is 
caiisd the section length. 

{00945 Speciffca / the delea on lOcS ^-d ci. a sute Ic ei ipo sen jn t ir e tiv, 

basis of he mpute" S0v.rd va eotm iom-t(,n S "x'" o mdloT v.fDtaiixd (ton \\-^^ Icula'^d 

value seqta to or e* {■^■^nthw eit i8\e! thiest-od ("H ^anceq^^a lO urnore thi-r the loise eve threshold TH,,) 
f |. (. I h r X t> 0 Of more tian a preset ength of time (heismatter called the m n mum silent 
<, 1 f 11th IM ^ 1 jmotse section length {DRS,;i(n))!s detected in this case the section )s detected 

as a silen; section or a noise secliars. 

[0096] Stnce a nomna! voico o sin announcer tn t nt^ws pro<)r im & oqu-j to SOoB o' mote tt o s lent bvc tircfet^oid 
(THj) ts sot to SOdBandtipmirmumsttpntspt'ion Mt,tf 01=;^^^ rssctfoO^so" ir wmbodinsnt Onths ot or 
haid smcethevoice evefofbact<grouidno(soma«pnrtp ogram when "pe-tt-tofb have qottcr n'o fi,!! swinij bijcc nes 
about -35aB. the noise teve! threshold (TH„) ts set to -35dB and the rrtinimum noise sec&on iength pfiN^jji) is set to 
1 .0 sec. in the embodtment. 

[0096] The storage unit 1 04 stores tne digitaf audio/video infonriation obtained and the time information for each of 
the siterti and noise sections detected by ihe detection unit 103. Tiie stomge um 104 also outputs im audio/video 
information to the reproduction unit 1 06 and the time information for each section to the control unit 1 0? in accordance 
with instructions from the controi unit 107. 

[0097] Tne operation tjnit 1 05 ailcws a user to instruct storage control of the audioA^ideo inrormaaon, instruct repro- 
auction of the stored audtoyvideo information, and enter a summary reDroducinQ time at the time of summafy repro- 
Quct 0"" The f^pG! ation unit 1 C6 outpt<*s these insti uctions *o the control unS 1 07 so hat ths cont'oi unit 1 07 wt!t cont ol 

each unit accofdinqly. 

[0098] ] oUiQitd iUtJ„\.' Jf>o ( otn ^ti-^- uu'rtmfUtiom h(»stofa f>tnt10^ ^ r ttf>(J '^thotopicuu fiiimt 
10- ^ie >i. o uc il. 1 JOil 1 06 s r( V, i f dv od<>i f 10 1 1. Jt (>U (i id J utf lt:.Xl^J --uUi Ji^o ! f ) sr-tto i tie 
viceo :n;ormaiion aiid Irse auoi-D infonnatson, av-.a oEtforms summary rearoduciton in accordance with the instructions 

[00 «] Ti<. itipf^d a lyi 'j,..^' -.10 viJ-^o-^gnaiistotht ciiii^it\ i-^ Ob 

[0100] Aj'hx in the T! ^d 11 n! 1 o rr^ - a dt-^cdc^ tt e d tddudtovdi. nc 

naEi-^n int( h vi nf m PdrJ he lu i n!j n rjf i s ui hetweoi *ho 'iideo nf rmalion -jnd tic 
r-ij m ^TH^ticn n dv d hie'^a wh-^n th ""it y 

[0101] fie r-ontto! ijH i HI' Cont dI& he -tc d'^..>jrjqice wtti n tijcto'^s tru^* -^d 

^ ^mthw f'^iot v< 1 * ■* 05 0 J^ici ^Ciq^st^ \ ^ imavfeo ^ ion or tco 

hclrrc(rfonnHtfon<^ntl 0 '^It-n^ -rc/ 1 q it 0% i'-o'^ iOvc ruf tit ^ 

nii-rsrto oft^if^r'^i: udu- !t.r yper t(on of t^e ie.5foductiin unit 'Ots or Lasis o' rformat oi an Shi, doc Qf-d 
seqinonrs (nereinafiercaHeo trie s>3qment informattonl 

[01 02] Tnr- p f^^.s=^ t ec t^-' digest segments to be extracted (berei'^aner ca led the digest segment decision 

process) \mH bo dcsc-ibod i^itDr 

[01 03] T'lc a jdio o gnals and the video ssgnais am inputted from the reproduction untt 1 06 to the display unit 1 08. 
The display utui "o? u spfays the Inputted video signals on a monitor scneen or the like while amplifying she audio 
signals by means of s speaker or the like. 
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[0104] Relemng nexs to FIGS. 3 and 4. the cigest seqmens decision process penofTtsd in iJie contfo! uni! 1 07 will 
be next dsscrtbsd. 

[01 05] Ir general ihc f udto n onndtior edaecf to the ai.d o/vidoo t ifonra. on p ays ar t ^portart ro e r su -nrrian/ing 
the audio-Video n'omation m setter tine {hrf-i the time length Df the audto/vtdeo fr'orndt(on 'ecorded o provided over 
a communfcatforss tine or tne ftke. 

[0106] For pxample m a soIpws on oroacirasting program f- notsc <5ecton mcJiCRtes ^n pxcrtiig part of tho piogram 
while a sitent section mcftcates a part that changes scene or switches program contents. 

[01 07] Specificaiiy if the program s *i sport watch nq prcq-am since responses fron^ spectators show n background 
noise such as snouts and cheers, an exciting scene wiil be much higher m sound levei than Ihe other scenes, ana tne 
Drt nek d the e><vi tng -ceie ca^ be rega'^ded as a feature oarl cf he viaeo ifoinat -^n 
[0108) Ci the 0 ner hard I'theproqian s a news pfogr<jTO since a si e""* sec ci oi so "affcd irit.'^af foajs-^) s 
.^M, <=■* i-t 'ITS cf bwiithrc. news contents aid the oart tiat foiows "10 cajse s^^ows ti-- r ext cot i,t :> k part 
V br - '"a* J " .Jdii of I'--' V liv nfort^a izi i sppctiilly fV p-irt th it to Iip «.(f£>r !or it-jv^^ i^p tii-T xi 
^ K rc>! ^LPi md^lL't) nvi^b - fch^*i=jniT>a'yof outtnjofiK Lonlon^ cot i^tne f 

[010Q] A-^ t\ ^f-i i\< Cv htip<!( ihai f cw-,*fto '■i i^'it s"( itcn b ■"ow , mpo'-ar! u jnjnn tio (.itMh prt 
^pW^] ore ^f>d nc T^t-^c i rj c Sv' ai^^o- -^^^ a cnur.trns'v,! i thf roi^e '^''^.^on -^i-^^tx 
position of the silent position and -he noise section relative to ;he feature oart of the audiojVideo information are dflerent 
'^tijrt^hotlv ^'i!Ltiieri'< tnopiccLfc r.. ■-^ ' c^-- ' IffoiM bt^tweot tht. stler> s v y 

fOIIO] Fjrtner s nentionea aliovo ^ince tnr' cf >" ^ \- ^ ^ <^now'? tic begir"" ng o tht, nest 

conlen s ssoecia ly snort s irnai i or -^nt ip ^ - ~^ , ^ g ver to iie digest segment 

decided based on the SI f>n -oction idn hat to thi, a qcs s",.- t acc^edba^-es cn {n-^-osesscion 
[0111] T>ius tie siif^rS st-o ior and he noise $< c- - '^s ^ o t *ur~iaton i,an he cnaractenzed on an 

indiviQua! basis r the e-^bodirent the c gest s-^g" «e"« c c-^vOi*. Js-vftieduulc^ by Dcaisofeshert'^esient 

[0112] In ttie digest seqinent cteasiors process of tiie emboGtmem. tne start time (51 soj). stoo time ii^bfefc:) and 
importance (l>^SS)o} taohoigsb^aeg'-^tnt diedocidod onli ob^siscfwhcShot tho dtjestb.E!5 Tier !i> n c- sn,nli.ect on 
or noisc section. In the toiiowtng description, V mdiostes that the section ts the i-tii siient or noiss section, and 'i" 
indicates ihs j-th digest se^ent. 

[0113] in the dfcjest segment decision orocess of the embodimsnt, the start time and importance of each digest 
segment ans decided on the basis of whetiier the digest segment is in a silent or noise section to fist digest segment 
candidates. The digest segment candidates ar© then narrowed down to decide the minimiim digest-sespr^enl lime 
length, the typicaf digest-segment time fenglh, and the maximum digest-segment time length so as to decide the stop 
time of each ot the nairowed-down digest segments. 

[01 14] Further, in the digest segment decision process of the emisodiment the section iength information {ORSSj) 
on both the siient section and the noise section is heid for use in setecling a digest segment from the digest segment 
candidates, in the embodiment, after the digest segments candidates are decided and narrowed dovsn, the stop time 
of each narrowed-down digest segment is decided using the section length information (ORSSj). in deciding the stop 
time to be described later, it is necessary to determine whether the digest segment is decided on the basis of the silent 
section or the noise section. The section length infomrtation {DRSSj) Is used for this determination, 
[01 15] Speofftcally, in the errtoodimsnt the section length of the target noise section is sat for the digest segment 
based on tfie noise section concemed. On the other !iand< DRSSj = 0 is set for the digest segment based on the silent 
section. 

[0116] in the digest segment decision process, when the stop time is decided in a manner d^cribed later, it can be 
detennined that the digest segment is set based on the silent section if DRSSj = 0, or the noise section if DRSSj 0. 

[Setting of Digest Segment in Noise Sectionj 

[0117] Since the noise section snows an excn ncj p.s;i of tno ciogr-ir ihs iioise section itseif becomes important, in 
trie embodimeriS, as shovvn in FtG, 3, tiie sta-' ccik^or: c-f seci:;--^ deteotsci by the detecticn unit IfJS is set 

:is the itart position of the digest scgmonr, 

[01 18] In a sport-vvatciiing program, it shouts, sr^-j chae-s -ro-^^ scijctators are coliecteci and the coiis^cteci sound iS 
contained as bacligfound noise in the audio intotriiat en -H'Jie^ to ri'o auciic'vidGo infoimation concei-ned it wili bs rncre 
effective in siimmary reproduction that the reproduc.ic-n i:urs fr;;-n a pa;l a bit previo js io tne exciting scene, in genera!, 
an exciting pail such as a good play and a goai or scoring scene in a spott gamehas sotna time delay uf iii i ihe spoctf-iors 
cheer over the exciting scene, that is. until the noise section appeafs. For this reason, the start time of the digest 
segment based on the noise section in the audio/video information such as on the sport-watching program may be 
moved forward At from the actual start time of the noise section. 
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[01 19] On the other hand, the stop time ol the Qigest segtrient in the noise section fs decided on the basis of tiie end 
position of the noise section. 

[0120] in view of the contents of the digest segment to be extracted, the end posftion of the noise sectioh basicatijf 
needs to be set at the stop tfrne of the digest segment. However, if \h« time length of tl>e digest segment to be extracted 
is Eoo short, the scene concerned may be made difficult to understand. On the other hand, unnecessarily iong time 
length could coniair» a tot of needless information, and an increase in information amount mak^ it impossible to som- 
manze the video irsformation unemrigiy, 

[0121] To avoid the above-rrjent'orjed problems, themrtmum dfgest-segnenttsme Jengsh (DR(at„). tiie typsca! dsgest- 
segneni time iengti p^-yp) and the maximum digssl-segmeni time length {DRy^J are set tn a manre' desci'bed 
iater for use m setting the stoc time of the digest segment. 

[0122) fzr e>';T!i:is -ss shown in FfG 3 when ihe noise section (DM te g the nofse ssrion a in FiG 3)) does not 
'eaot- r-^t, ir.r^:,-num diges'-saqr^eit 1 me lerqth fDR^ Ihe 'ine iengt^ of ;he digest SDymsn; \s the Tituim a.gcst 
reqns it S'Tk o'^-y.h {i>\<ii^) "he Tiirif^tjir o g>ist se'-jr-c"! U'ms a'-g b (IJHy,^- li added 'o \hc sta't tints' :>{ tiip i-.g-O: i 
segment, and tne resiiitJjriE tirr^e is sc;! for mo stop iinse of tt'.'i digest seymont. 

[01 23] W^ion ;h£' nciso section (DN, (e g the noiso section b m FiG 3)) ta eqti-ii to or more ihan tho mmim jm airjt-dt- 
segment time length and equai to or less than the maximum dlge^-segment time tetis^h pBnijax* the noise 

section length is the time length of the digest segment, and the stop time of the digest segment is set at the end position 

of ihe noiss section. 

[0124] fjrt ivs wo-' I'K^ \ .-^ a ■ D ' ) 5, s.CL\..>^ c tn "!G 3}) exceeds the maximum digest- 

segiienumc icngti p^f,<j^ , scg-'-'O'' l-^?c'g"~ is added to the start time of the digest 

segnsnt and ire resuitan; i nf^ is se; for ihe stop ;imo o' tf-e dtgess segment 

[01 25] In other words, the slop sime of the i-th digest segment tn the i-th noise section ia detemnined from the segment 
time iength (DRDNj = DRSSj) as follows: 

ffO<ORSSi<DRt;,i„, 

SeSS> = STSS*DR,.„. (Eq.1) 



tfDR„j,<DRSSiSDRj^, 
SESSj » STSS f DRSSf. {Eq.2) 

ifDRjy^,<DRSSs. 
SESSj = STSS + DRt-yp. (Eq.3) 

it shQutd bs noted that when the start time of the digest segment was moved f onward At from the start time of the noise 
section, At needs to be subtracted from each of the minimum digest-segment time iength (ORj^Sd), the typical digest- 
segment time iength PRyyp), and the maximum digest-segment time iength (DBi;,^a)t) so thst the time iength of the 
digest segment wtit be consistent wth those of sse other digest segments. 

[0126] tn the embodimerit, the stop time of each digest segment is set for the digest segments that were narrowed 
down from tho digest segment candidates in the process to narrow down digest segment candidates to be described 
later in ctl^er words, the siart time of each digest segment is set on the basis of the noise section to iist digest segment 
candidates, then, the process lo n arrow down the digest segmem candidates is performed in a manner described later. 
After that, the minimum digest-segment time length {DBMin)< the typical digssf-segment time iength (DFtryp), and the 
maximum digest-segment time length (DRj^^j.^) are set to set the stop time of the digest segment concerned. 
[0127] On the other hand, the importance (iPSSj) of the digest segment in the noise section is set using the section 
iength (DRDN;) of the noise section. The longer the section iength of the noise section, the more the importance can 
be set. 

[Setting of Digest Segment in Siient Section] 

[01283 As mentioned above, since the siient section shows a scene change part or a part that switches contents, 
the part that toiiows the end of the silent section becomes important, In the embodiment, as shown in Fig. 4, the end 



10 



EP 1 271 359 A2 



position of Si stierit sectior having a sediofi length (hereinaftef called the arfditionai minimum sii&nt-section length 
(ORSAjy^.^) emat: or ntois preset forthe siient section detixrted by the detection unit 103, for instance, 1 .0 sec, is set 
lor the start time (STSS) of the cJigest segrrsent. 

[0129] Of course, the silent section could bo of iittle or no importance. To detect a part in which there is an obvious 
s "pause" that ensures the occurrence of a change sn contents, the additional minimum sifent-sectlon length (DRSAj^jn) 
is laid down in deciding a digest segment so that the end position of a silent ssctiort havfing a section length equal to 
or more than the add^tonal minimum siient-sectlon length (DRSAMm) wilt be set for the start position oi the digest 
segment. 

[01 30] On the other hand, the stop lime of the digest segment in the silent section is decided on the basis of the start 
10 position oi the siient section that follows the silent section used for selling the start iime of the digest segment, 

[0131] ir Shis case: the section length oi the silent section that follows the silent section used for setting the siart 
time of the digest segment does not need to be equsil to or more than the adcJItion^if minirr um silent-sscticn length 
(DRSAji^ij,). Therefore, all the silent sections detected by the deleciion unit 103 are searched, 
£0132] Lilce in Ihe noise section. th& stop iime of the digeat secjrrient is set m a m;inniL'r desciibed if-ier i;3:ng the 
W minimum d!9f>s; -s--5gmj]nf time lenc]lh (HRf^jj^) thotypica! dif)(:-.=5l -.swj-rif'r;; t;me length (DR-T-yp). find the rnfLvimurr' dif^e.'il ■ 
segment lime length (DRj^^.^}, 

[0133] f-'or ex.^mple. as shown In f-IG 4 when -ht; stKrt co.s t^on oj tne stienj section {0S^^■^ (i? g., the silent S'3<::tion 
a iri f-lG. 4)). \Mti!ch is detecte!^ immediately nKe- e siien: secnot; set as the siai't tifns of ihe digest segment, does 
f:o:. reach ihe minimum dlgesi-segrnei'i Jr •.; e iyU' iD>^y lime lengit- o- I'-.a digest segment is the minimum 
20 digest -segment time length {DRj,<|,,1 ~hf- "i::-:n.;:v c.003; .scgmsn: i mo ienc-h tDR^j,,) is added to the start time of the 
digest segment: snd the resuiiani time is &ei ?or ;h8 stop lin^e c; - he digest segment. 

[0134] When the start position of the siien; section (3S., ^ (e g , ihe silent sect:cn b in F-G. A)) wh:cri is detected 
immediately after the s-lsnl section set as the siait tinte of the csigest segmsni, exceeds the minimLitr. dtgc-st iogmcnt 
time length ('DRiy,r,) but does not reach the n"';a.'<imii!r digest-segment time iengih (DRs^-i)- the stari potiiion ct tne 

S5 defected siSent section (OSj ,1) is set for the stop time of the digest segment. 

[£J135| Further, wnen the siart position of the silent section (pS-^.,^ (e.g., the silent section c tn FIG 4)!.. which is 
delected imniecl:aiely after the s^leni section set as the staj t lime of the digest segment, exceeds t;ie maximLim digest- 
segment time length {DR^.to<)- iengih of the digest segment is the typical digcst-scgmant time length {DR|^), 
Ths typical digsst-segment time isngth (DR^j^p) is added to the staft time of the digest segment, and tha resultant time 

30 is set for the stop time of ths digest segment. 

J01 36] tn the embodimsnl , when the stop time of the digest segment is set using the minimum digest-segment time 
length (DRyin), the typicai digest-segment time length {OR^yp), and the maximurti digest-segment lime length {DR^jju,), 
the next silent section is detected in the foflowtng sequence, 

[0137] The sitertt ssetioh {DS^^,} that fofiows the siiem section used as refemrtcs to the start lime of the digest 
as segment is detected in the foBowing sequence of operations. First of aii, it is detected vs^hether the start position of the 
silent section (DSi^.) detected immediately after the silent section (DS[) is equal to or more than the minimum digest- 
segment time length (DBf^„) and equal to or less than the maximum digest-segmem time length (D%aj^). if the start 
position does not exist within the range, it is then detected whether the start position of the silent section (OS^i) 
detected immediatefy after the siient section {DSj) exists within the minimum digest-segment time length (DRfjcm). If 
the start position does not exist v¥fthin the rang©: the silent section (DSj^^) detected immediately sftartiie siiet« section 
(DSj) is detefmlned to be in a range ot the maximum digest-segment time length (DRy^jJ or more. 
[0138] In other words, the stop titne o! tiie j-tti digest segment in the i-lh silent section is determined as follows: If 
the start position (ST) of the siient section {QBi^^i was foijnd in the section [Di\,jf), OR^a^], 



SESSj = ST. (Eq,4) 

[01 39] If the start position (ST) of the silent section (DSj.^^) was found in tha section [0, DRjjjj„], rather than the section 

mm.- E)Rto]> 

SESSj-STSSi4^DRs«j^. (Eq,5) 
[0140] If the start position (ST) of the siient section (DSj^..,} was not found in the section [0, DHf^^, 

SESSj » STSSj 4 DR,j-yp- 1^'^-^) 
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[0141] In the seqiience of detection of ihe silsnt section (DSj^.^), even when the next sHsnt section {05, t exists in 
the mintmufn digest-segment time length (DRj^j;^), If She statt position of another siient section (e g., DSj.^j,, where n > 
2) is equal to or more than the minimum digest-segmsm time iength (D^jj,), and equai to or less than the maximum 
digest-segrt?eni Eime length (ORj^^), the next siient section (DSj^^) that exists in the mtnimunn digest-segment time 
iength (DRj^sn) is not handled as the siient section that foiiows the siient section (DS,) used as refensnce to the start 
lime of the digest segment, and the siient section (DSj.^. where n > 2) is regarded as the next siient section (OSj.^^). 
Thus the stop fime of the digest segment is decided on the basts of the stfent section (DSt^^ concerned. 
[0142] Like in the setting of the step time of the digest segment in the nose section, the stop time of eacii digest 
segment ir? the siient sectson is set forthe digest segments that were narrowed down from the digest segment candidates 
in ihs process to narrow down digest segment candidates to be described later. 

[01 43] Or»the other hand, the importance (tPSSj) of the digest segment in the silent section is set in the sanfte manner 
as in the noise section on the basis of the section iength {DRDHj) of the silent section. However, since the siiertt section 
Is of mors importance than the noise section, it is tSetemriinsd, for exampis, by the foliowing equation 7; 



in the equation 7, f(») Is a weighing function, and in the embodiment, the following weighing function is used: 

f(x)-ax + b (Eq,8) 

in the equation 8, a and b are constants, and thefoliowing specific example can be considered: 

f(x)-x+100 (Eq.9) 

[Process to Narrow Down Digest Segment Candidates] 

[01 44] The summary reprociiiction process to be described later may be performed on a!i the digest segments decided 
as mentioned above on ihe basis of the siient and noiss sections. However, ihe digest segmer^s to be ae! are narrowed 
down for pu-postas of reduction in amounts io be processed and prevsnOon of reproduction of unnecessary digest 
segments, that is, prevention of reproduction of inappropriate digest segments, which meat» that even the digest 
segment of KUte importance could fee of increasing importance in the merging process to be described later 
[01 4S] in the embodiment, the process to narrow down the digest segments is carried out from the digest segment 
candidates listed by the following equation 10. 

[0146] Assuming that the time iength of all the digest segments is the minimum iimit time {ORjjj,j„), the equation 1 0 
is to compare a multiple {e.g., K-t = 2) of the number of digest segments to be narrowed down with the number of digest 
segment candidates so that the smaiier number will be set as the number of digest segments. 
[01 47] For example, If the number of listed digest segment candidates is (N Pea) and the digest time is S, the number 
of digest segment candidates {NPn^^) to be newly set is obtained as; 

^Pr^v - Min(int{k, x (S/OR^m,„)), NP^) (Eq.10) 

in fheequaUcn 10 t^1 :sa corsart, Winfa b] meansthatsmaiieroneof a and b is selected, and int{») means thattne 
fractionai poition of the number ;s d'oppsd Funher, NP^^^ represents the number of digssi segment candid^ites a^se' 
narrowed ciov<»'n, and the DR, represents the minimum limit time. 

[0148] Tne minimiim lim:l Uvs (DR^ j;.^,^) is the minimum time necessary for a person to understand tns contents of 
a digest segment. For example, -.rt the embodiment tne n~::r:n-;uii-; ; ■■-e iDR^yi,^} is fouf seconds, 
[0149] When the number or digest segment Crindidates -n^js c.n:;:ji«;ec; s larger thc-in tne muitipie c-f ihe rumbe: oi 
digest segments to be narrowed down, ■^al is v/-iCi- ^-r.^,. ?, ■•^-r^oo^ o'- a c^at segment cartdidatos corre- 

sponding to the number fiP^^, -ire s>?ler1f o r c:.?s.r-"n:-;n . an ''■;nor..--. v::£: ana tt-.? others a-e deleted from the 
list of the digest segment candtdaies. 

[0150] In the embodimeril, Ihe digest segment candidates are thus nasrowed down so ttiat £^ie slop time of each 
digest segment is set for the narrowed-down digest segment candidates according to the above-mentioned setting 
method. 
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(Setting Qt Minimum/Typicai/Max imum Digest-Segment Time langthj 

[0151] As discussed above, the digest segment to be extracted Ms a time length as tong as possibie so ttiat the 
digest segment wii! be made understandable. On the other hand, unnecessarily long time iengtt^ could contain a lot of 
needless information, and an increase in informatson amount makes it impossible So summarfee the video Infomiatlon 
unerrfngV Therefore, in {he embodiment, the minimum digest-segment time iengJh tDBiysn), the typical digest-segment 
time isngth (DPi^yp): and the maximum digest-segment time length (DR^ja,,) ara set in a manner described faefow, 
[0152] Forexampts, In the embodiment, themlnimum digest-segment iimelength (DRfjin), the typical digest-segment 
[ime length {DRtyp}: and the maximum digest-segment iime length (DRbiax) are determined by the foilowing equations 
so that the contents of each digest segment to be extracted vM be grasped unerringly, 

[0153] Considering ihat the digest segment is made easily visible to the user, she minimum digest-segment iime 
length {DRjy,jf,) is set as shown in ectuation H so that the digest segment will have a reiativeiy long time length. The 
typical digest segmsnE time ler^gth (DR^^p) and the maximum digsat-segmant time isngth PRsfa^) calculated tjy 
muitipiying the mtf^imum digest-segment time length (DR^iir,) csfcuiated from the equation 11 by a constant as shown 
in aquations 12 and 13. 

DH^^ MaxiDRLMjn, (K2 >= |S/NP^^)}) (Eq. 11) 



{01543 Here, Kti and are proportional corsslants, ana Max(a, b) means that the iarger value out of a and b is 
selected. Further, Kg (a 1 ) is a coefficient for use in deciding the minimum time of each digest segment. The larger the 
value of % the iongerthe minimum time and the waller the number of digest segments. For example, Kg ~ 1 , Ky^ - 
2, and Kyg - 3 in the embodtmem. 

[Merging of Digest Segner^tsl 

[0155] Ir the embodimerit, v^hsn two or more Oigest segments coincide with each other, the digest segments are 
merged into a digest segment. In this cKse, the importance of the digest segment generated by merging two or more 
digest segmenSs taltes the highest value of importance tiPSSj) from among values for all the digest segments (see the 
foiiowing equ-iitfon; 14). 

tPSSj = Max{fPSSj, !PS%f,) (Eq M) 

Further, If STSSj < STSSj^^ and SESSj > SESSj,,n for two digest segments SSj and SSj+n, the fbllowing equation Is 
obtained'. 

SESSj^SESSj,„ (Eq.15) 

[01 SSJ Thus, even wherj a digest segment is of little importance, if the digest segment coincides with another digest 
segment of much importance, the digest segment of ilttis importance can be complemented bythatof much importance, 

(Decision of Dlj^est Sej^ment] 

[01 57| In the embodiment, the digest segment candidates are selected in descending order o! importsnce to achieve 
the specified digest time in the final process, 

[01 S8] The selection of digest segment candidates is continued until the total time of the selected digest segment 
candidates exceeds the specilisd digest time, 

[01 59] When the digest segments are decided in descending order of importance, since the time length varies from 
segment to se^er^t, the total tinrw of the selected digest segments may exceed the spedfiad digest time, if exceeding 
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the specified digest time becomes a problem, necessaty measures m\\ be taken against ths overtime, sucf) as to share 
the ovetttme amoBg the decided digest segments by eiiminating the sharecJ time from the stop time of each digest 
segment. 

[0160] Referorg next to FfGS. 5 io 7, the digest segment decisiort process in the summary reprodiicing operation 
of the controf unit 107 mil be described. 

[0161] FIG. 6 is a ttovwchart showing a digesi-segment decision operaison for summaty reprodtiction accordirig to 
tne embodiment, F!GS, 8 and 7 are ffowchatts showmg setting oparations on the stop tima of dtgast segments decided 
crt the oasfs of the notse section and the s tent section n the digest-segnent decision process respsctiveiy 
[01 62J AssiiT {"at 'he audio/vtoeo n O'malson requtrea ^or surma7 rep'-OQUction ts alfeady &.o)ed r .he sic age 

{ ^ the wDerM on is cajried uut v^hen ihe ise irslrucls .he summaty sep'-oducjrin 
[0163) '^s siu'v'' r J 5 //hen the ^ser e^^ters an insf-ucttor forsummarj? te^>rodJC^ Dn th'-o^q'' *hc oDtrn on ami 
10c h^->ccn ^ Ltil 10' dete mires Ahethiji 'hs Allien' and nose section detection process is ptrfc n on tiiosptv 
f -^u d r vd-^o t! h fiv ton 'o! h" f it i rro t. pp S11 if it ts ootermned that eslonl and T-iiS' i:p..t(c h-v^u 
^ LViOL-^ \ J ■''•^'■'•{.ci toi t X' ujio'v ■■u^in'Mn tiof :;oncu 'ieo t^cddtaafonaadoutoftf i^^btoicsgeL' {i.*OD S1^i 
[01S4j O^n-^. hecM'H { NO'S tMf ^ iTjisE -s'^ro!- j^"'>'^tof%!tor->s<->vi'vPO'bt>M i>e'-'.;it-oicntt ^sovr' \i 
rucn/vnoo TTTidtJon yc> .h^cont'c ni " "I'^crn 'ols f! e dctcc tor f IOC f^deircf stferldnd r((ii';op?ctnns',om 
tne specifier audio/viceo imotmahon ^ctesstficsffon js-eo is\w &13)}. 

[0165] 1 nep ihc CO 5tr tut 'D7fjtv,K&r v, ^-^^ ^ i 't. ct < ^lObD' Jioos^ im.>!.5t£.pS ■i' 'nJ 

Srtt^dt ii.tff':i Jijcfc fccgn-e lUrird Jj^--t ' ■^ ewdojli; vijs^sf r-^qc'i l iC + ^dtN i& 

step (Siep S16!l 

[01 6^] Srenffca / tie telr-i" an-i (^n- .-^ o > --e'^^iyn hti\, rg ,he additiora mtrcnun stism sect on 

iei g h(DRSAn,^isn„ h_ ■'osesecti^-' ' ^ > tine and impoftancs of each digest segment a>-e set 

[0167] Tiecortc intt ' .hei ►J.-i'!. i <, ^ \^:^• <: '^v down the digest segments from the dtgest-segment 

candtdaie iist crearea in step 3ib (decision s;eo isiod blti)> 

[01SSJ <. Ill w t^-" ttj ( f'J^s 1, ' . Cs-,<.d d'^v\fi -torn *he IIS ^^digf's* '-sa^if't^ c^^nddites 

Sc.-?! uiiEtrloi lix.T <^!-.o theinouter^ v,est'inL no honnim nMir-'ittmc (DHj,,.,, ^nd cairu a<£d t u-^ibe) of 
c. gosit 1 :> c c ^^,L^.,ted fio ii tl^s:. listed digest segn^e ttcandidrftus in duscord itj ocor o' tiieo'"arct. to nar.ow 

aown The aigeai aegrt^srn candidates, 

[0169] Then, iha control unit 107 caicuiates the minimum digest-segment time length (DRj^,:,,) on the basis of the 
rumbof of digest segments narrowed dowrt in step SI 6 and the ninimum limit time iDS\M,n) -^ifd se s the typical 
digest-segment time length (DRtyp) and the maximum digest-segment time length {DRititaj,)on the basis of the minfmum 
digest-segment time length {Di%nQ {step SIT). 

[01TO] Then, thecontrof unit 107 deteirnlnes the type of sound section, set in stepS15, of each of the digest segment 
candidates narrowed down in mp S1 6, that is, whether each digest segment is set or* the ijasis of m& noise section 
or the siter»t section (step SI 8), 

[0171] Specificaiiy, the determination is made by the value of the section length of the siient section or the noise 
section {i.e., whether DRSSj - 0 or not) on which each digest segment candidate is based, 
[01 72] Then , the controf u nit 1 07 sets the stop time of each digest setprtent candidate according to the type of the 
sound ssciicn (decision steps (steps SI 9 and S20)). if the digest segment candidate is based on a noise section, the 
stop time of th& digefst segment candidate wiii be set according to the end position of the noise section (step S19). On 
the other hand, it ttie digest segmcr} candidate -s t>ased on a siient section, the stop time of the digest segment 
caritJidaU; Is set accoi-ding to the .star; LXJ.-jit ic;;! of sriott-er silent section, which was detected ttTsmediately after the silent 
sectiof! JSGC fis fsferencQ to ttie start ir-^ istep S5'"V 

[0173] The processing operation on the stop time of e«ch of the digest segment candidaies to be set on the basis 
of whether the digest segment candidate ;s in tne silent section or the noise section will be described laist. 
[01 74] Finally, ttie coritroi ufjit 1 07 merges iwo cr r-^ors digest segment candidates ti^a; coincide with each other in 
the above-mentioned man ner, an d sslscts d igest segrr^ent candidates to be extracted in descending order of importance 
so that the total time of the selected digest sogmsnj candidates becomes the digest time inputted in step S14, thus 
aeciding the digest segments (decision step (step S21)). 

[01 75] After ccrnpietlcn of the selection of the digest segment candidates and decision of the digest segments for 
sunir^Vipy reproduction, the ccrttroi ismt lC7contfoi6 the reproduction unit 106 to start the summary reproduction based 

on the dedded digest segments. 

[0176] Referftng next to FIG. 6, description will be mad:; trs crj.isssing step S19 of setting the stop time of 
each of the digest segment candidates generated on the br,£ s c tre n.j:£e sectior;. 

[0177] It is ffEst detetmined whether ihe section lengtti (DRSSj) of the noise section on which the digest segment 
candidateis based is within the rnaximtjn^i digest-segment time length (0%a)() (s^«P S31). if the section length (ORSSj) 
of the noise section exceeds the n^axiintjm dige-st-sagmeh: time length (ORwax)! the typical digest-segment time length 
(ORryp) is added to the start position (STSS) of the noise section concerned, and the resifitant vaiae is set as the stop 
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time {step S32), 

[0178] On {he other hand, if the section fength ol the noise section is shorter than the maximum digest-ssgment time 
length {ORjijaj;), it is then determinecJ whethsf rhe secrion lerjgth of the noise section is iongerthan the minimum digest- 
segmerit time length (DRj^jn) (step 333), if the section length (ORSSj) of the noise section concerned is Ionger than 
the mfnimum dfgest-segment time length (DR^jf,), the minimum digest-segment time length {DRt^sn) is added to the 
start position (STSS) of th© noise section concerned, and the msuitant vafii© is set as the stop time (step S3S), 
[OITSJ Refsrrirsg next to FIG. 7. description will is© made about the processing step S20 of setting the stop time of 
each of the digest segment candidates generated on the basis of the siisnt section. 
[01 80J Fj rst, the next sitens section that follows the siient section concerned is retrieved {step S41 ). 
[01 81 J As discussed above, even when the next sileni section exists within the minimvim digest-segment time length 
(DRyiJ. priority Is givan to any o;nef stien; section tiial ts equal to or more than the minimt-m digest-segment time 
length iDRf,^:r.) ^-^"^ -'^^f*' '° ^^^s than the maximum ctigsst-segmeni time -sngih -D^g^). Thefsfore, when the next 
fiisnt seci:on exists within the mtnimum digest-segment lime Scngih (Df\,|p), !he first silen: seciion thrat exis.ts beyond 
tiKi mtfiimum digest-segment time lengtii (ORfynf,) is ;ilsa retrieved. 

[01S2j It is next tietermir-ed wlisther the; lime length (ST) to ihc. i5ta;i position of t?-;o ?5ilem seciion (na. , ). wtiich was 
dcLected immediately a-ier the tJilent sedton [DS;) sel as Ihe start lime o! the digest segminiit: is equal to or more than 
the minimum digest-segmenl time length (DF^,;,,) and eqsja: lo ni less than ihe maximum ciigest-3egm:?nt tme length 
(DRfij,^) (step S42). If the lime ierigth (ST ) io Ihe stajt position of the silent section of the stient section (li)Sj^<) is equal 
to Of mote tt;an the rninimLfm digest-seg-netu lime lefigtti ;DR(^<,,.J and equal lo or less than the maximum digest- 
segment time length {0%,<rjj): the time length ST of the s;art position ;s added to the start time (STSS,) of the digest 
segment, and the resiittant value is set as tne stop time (step S43). 

[01 83] if the time length tST) to the start position ti-e si:e'-:t section ^DS, , : is ro: oqua! to or more than the minimum 
digest-segment time length {DRy.r,), and not ecu?.: to lesi than ti-t; ^-^-ay-rwr t::Qos{-5egment tinna length \ORjsia)i)> 
it is then determined whether the time length iST; sc me siart pcsT:on o? -.hE siionl section iDSj.,,) detected immedfatety 
fiPie! -he siient p.-K;t;cfi tpSji is. shorter tnan the nvr-r^um d^gsst-jegmen; -ime ls;ngth (DR^^) (step S44). It the tirne 
length (STt torhe stan position ss siwrterthan the minimum digast-s&gmcnttime lertgsh tDRf^j,^), the mtiwrsum digest- 
segment time length iDRf^jn) is added to ttic slart time .S"'SS.} of the digest segment, and ihe rssuitant vatue is set 
as the stop time (step S46). if ths time Icfsgiii (ST} to tiic start position is longer than the minimum digest-segment time 
length {ORsjj„), the typical digest-ssgmeni time length (DRfyp) is sdded to the start tim© {STSSj} of tha digest segment, 
and the resuftant value is set as the stop time (step S46). 

J01843 As discussed above and according to the en^bodiment, digest segments to be extmcteti are decided on the 
basis of the silent and noise sections delected according to the sound ieveis of Ihs audio/video infomiation. Therefore, 
summary reproduction can be performed on exciling parts and parts that switch conlenls of the audio/video information. 
Further, since the impiSftar^ce of each digest segment can be decided on the basis of th# section tettgth o? the silent 
or noise section used as reference to the decision of the digest segment, digest informatfon that enables the user to 
grasp the contents unerringly In a short lime can bs obtained 

[0185] Further, ths start time of the digest segment can be set at the end position of a silent section, wni:e the stop 
hme of the digest segment cencemea can be set on the basis of the next stfent section detected immedtateiy after tne 
start time of the digest soqmcri Theiefore the d gest segment can bs extracted at sjch proper timing that the user 
w ii not fee something s wrong <?f afi such as a part that shows a foaturo part c* ho autin/v 20-^ n on^at or c> ? part 
mat !S a good place to seave oft. 

[01S6j Forfienmc th ">taitt n > of p-fUivt.Je> n o'm^ityti i.ntt be bc it h' -,t -n t'l i,it i r ar -k, ise ,.-fUfu v'v'-iile 
tno stoD line of th(^ '"Mtiih t f -> - Ct £ c^-^' - i * c ^ -^-^^^j *ho rciso s^^-t "^t t.'e or<^ 
tne dig-Jit iP' 'ji^nl < >> \ < > triti- ■< >m} f r f:*' ir \\ 

tnat the user witl not feei somctmna wrong at an. 

[0187] lr-^d"ifcn th^, lop f ie oi cac^ o q< < o on the basis c nemmiuidigcs stgmcfltimc 

e>qh theivpc<3id ost scgre it lim-^ e'^g h ^ " - < - -'rest sogment time length T ereforo a time length 
enough fc tie I. sor to jndeis a-d tne contents . , ^ . j qest segment car be secLred white p'eventing the 

t me iencti c tie Ingest segment f on becon .„ —^t.- -^c 

[0188] Ai llj^ It ^ ^ '■^ : ^ d bass o' tie i/tdeo nornaton 

comciv-cdc dig ?l!j 1 <; *■ c - fo ao.. -c - o , , ^tori>fcvi cd .^ra 0:5 ;> anal=! 

[0189] Fjrthe ilae Lo men a ro e s th e£h(jid TH^) i u^^a c dwt'H.t he noisesr^ctrn uu tw-^ Ti-re 
noise level thresnoio (Tl-i.,i mav be used. 

[0190] h t^c Ms^ c ■■ ^ < "-^-K. ^ELM-.^.^ett-^tn'^i-C'^ej yns 

1 c rd^ lespectiVL! V ''Litt ripDroj vit^ Lmfrntv opiodLCioncir. ^ ^ - t^-m) c --t^. the case whet ncJ jt-^i 
segment is created frcm a noise section. 

[01 91] In other words a very exciting par" of the audic/video inform Kior t^c =5otiH h-^ el 0' wich exceeds the noise 
level thresholc 1 (TH^^). ts eetected as a noise section . Then tne imoortance of the aigest segmenl m the noise section 
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is set more than that of another digest ssgment decided by the noise \evel threshoict 2 (THf^), by means of a weighting 
function or the ffke as used for setting the importance of a digest segment decided on the basis of the siient section. 
[0192] As a resoit, any (mportam part in the audio/video information can be set as a dfgest segmem unerringly, whiie 
th© noise section obtained by the noise tevel threshold 2 {TH„2) can aiso be set as a digest segment candidate. This 
s feature altows the user to have a wide range of digest segmenss to choose from and perform appropriate summarj' 
reproduction. 

[01 93J Further, the above-rrtentioned merging of digest segments that coincide with one another could msuk in the 
merging of a very sxciting digest segment with digest segments before and after the vary excising digest segment. This 
merging process maizes a aicest segment of extreme importance, so that the very exciting p^irt can be replayed for a 

10 reialivsly iong time ai the rime of digesi viewing, thus perfomiing appropriate sumnaiy reproduction. 

[01 94) Furtnermcre, a CDnventionai CM (Commercials) cutting technique may be employed in the CiTibodii-nent, i'he 
probability is gereralsy h.g'r. that CM parts of ttie audio/video information will be noise sections. Therefore, if the CM 
cutting tijchnique is. combined with ihe embofiiment euch thatthe CU parts are detected before noise and siient sections 
Eire dot.jcted f-om the aLtdicViaeo ififo-rtiation for summary reproduction, an appropriate noise ievei thrsshoid orthnssh- 

1S ciidf. cAn bfi f f?t whici^ niskas it po.ssibte to poriotm more approphaie stimmary reproduction. 

[01 95] f-'or ;ht? CiVI cu!!:ng techniqija a msthod arc cievics ^c; s.iifnmari/mg video descfibsti in Japanese Laid-Open 
Paieni ApplicKiion Ng. Hei 9-2' 9335 ;S used, 'i'nis ;a;ct^n:qi;f: ;,=? ic deiec! a psti (ciip) that shows an enonrtous change 
it! contents in the video inlanti.^tion af!d Siicnt secii-jfis so '.ha, tiie CM oan will be cui ysing the clip .qnd the silent 
sections. 

20 [01 96] Fjrthermofo, in the EsnbocVnent, digest ssginarts ' c;ose pi'cxni'iiy \o one a.iother on tho ttms hx\s tv-A-/ oo 
merged. For example, a ssquencs of moving pictures SiJC" fi* WFEG p c;.;'t;,-; ^i^v i^^o -iniraiosesk required positions 
on the time axis at the time of summary racroduction, causing a proDlem ot iesnporary rep=ay stops durif»g seek time 
between digest segments at the time of summary* rsproouGtion. Th-s problem ts offensive to tne user wno fs viewing 
ttie digest repfay. To avoid this probism. aftsf ;he compietior ;ne r.Uovc- :-iemioned seiection ot tiie digest segments 

S5 ta extracif^d. digesr f egmentf in close prox^j-'-^ry to one ,sno!'-;s! on tne time axis are further merged into a digest 
sorfrions So tedijoo the niiribor of d^gost sognc-iis rocu --an --n '.no t-me of summary reproduction, so thsa the number 
of seek times is leduosd thofoby providing an sasy-to-view digest replay. 

[01 97] Aithough in the embodiment the detection unit 1 03, the reproduction unit 1 06, ar)d the contro! unit 1 07 operate 
in the summary reproduclion process, a program for the summary r«^roductton process may be read out via a computer 

30 to perform the summary reproduction. 

[0198] tn this case, the control unit 107 is provided with the computer that ioads and executes the program. The 
decoded audio/video Information is inputted into the computer, and silent and noise sections are detected from the 
inputted aucfio./wdeo Infon^atlon. Based on the silent and noise sections detected, digest segments of the audio/video 
infotmatioh are decided so that the summasv reproduction &f the inputted audio/video infomiaiion wii! be petfermed 

35 on the basis of the digest segments decided. The use of the program and the computer can dispiay the same effects 
as the above-mentioned summary reproducing apparatus. 

10199] Further, aithough sn the embodiment the summary reproducing apparatus 1 00 is constituted of the detection 
urtst 103. the reproduction unit 106. the control unit 107, and so on as mentioned above, the controi ut^a 107 may be 
Diovided vferti a computet and a storage medum such as a ha d disif !t th>3 configuration, a program that performs 
4S c oco'^'sm') CO ''o«pondir-t^ { {i« oporat oi of oach unrt cf tne summary roo oducing apparatus 100, such as the de- 
■^^1. A) U1I a I ( t(>>j(i.d ct urif 1 06 and the control un(t 10 ' i^ «lored on the storage medium and loaded on 
* i c V''' i" I. 'riti Ki cd.^'-i J ii of iK'bi'i ) iv odu ng<jppB(ra*us 100, such as the detection unit 

103 *h-^ rap Drtu- 0 1 Lnt 10f; 5-3ifiLCin s * '^■^ vv c e'^^^D 

[0200] r !i f ^ (> < ;) , < f s'j nniary reproduction process are 

^0 o^m^^d t e ptoQtcjr ' nn '^r f e rs,mt.i.^ v d .^j. t. jr ucneu operations of digest decision and 

summary rops'ocuclo 5 re.!- !ntt«3C''so ihcconlroiuni 1 07 tons>ttutcs the detection device, the generation device, 
ana the decision device according to the present invention. 



50 Ciaims 

1 !\\ro- 1 crmatior summarising apoa ?ti.^ V -^^ v " ■'ot wpppces of partial vid6oin{onTi.Viior> as 

some oarts of Video (nfomatton oased on auoio inton^iation irom the vicseo infofm.qtion to which the audio :nior- 
matton is added so that digest in'o mat or si ■^ t ic <f r^n t! e v deo tnforrratton wtii pe genei ated 

55 from the viaeo informatiori on tne bfisis of me partsai video infamiaiion extracted, characterized in that ttie ap- 

paratus compnses: 

a csassificaljon device 0 03. 1 07) whfch dassifies the vioso information into piutai sound sections on the basis 
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of the sound teveis In the audio information; 

a decision device (107) which decides the partiai video information to be extracted on the basis of at ieast 
either the tims-base position or the time tength of at feast any one of the plurai types of sound sectiofs Oassif ied 
in the video (nfonriation; and 

a generation device (106, 107} which extracts the decided partiai video information from the video infomnatlon 
to generate the digest infomiation. 

2, The video information summarizirjg apparatus {1 00) according to claim i , wherein 

the decision device (107) decides at least either the start time or the stop time of the partia! video infonnation 
on the basis of at feast either the time-tiase pos^ion or the time fength of at least any one of the plural types of 
sound sections ciassffled in the video information. 

3, Ths video inlonr-ation sumrnaivi-ig apparatus {'00'-- according io clairn 1 or 2, vvhsrein 

Iha classification ■iavice (1 03, 10?) cisssiiies in the basiij of the aound levais \hQ video infofnatian Into at 
least «>tjfirii«!ss. siiant s^Dtmns rtwi fioif,« section?. !h-iT 'al witnin pre-i(A rang;? of sojnd ifivs:-!:? 

4, Ths video inforiTiatlon Kij^;-nati;<i:Xj apparalus {' 00- aozord ng \o claim 3. wnf?r!;i!-i 

the d£.>cisic:5 devict; ■ 1 07) seis '.'ne st;-'" ;imo of iho ps i;s: v ic;sc; :-i'c: r-ii,; .y ;ii a timc-base position that shows 

the end of a cortesponding siien; section having a preset tirrie length. 

5, The video information £um.maf(2irtg aoparatus {1 00) according to claim 4: wherein 

after setting the stari Oms of the partial video information based on the silent section, the decision device 
(107) sets the stop tine oi the partiai video infonnation based on the time-base position of another silent section 
detected iirmediateiy after the sitent section concerned. 

6, The video information siim.'nafi?!:-ig appsrstus (1 00) according to csatm 3, wherein 

the decision dsvica {' 07} sets the slait time of Jiic ta: t;a; v.deo informaiion based on the time-base position 
that shows the start of a noise section having a preset t:rne iGngl fi. 

7, The video infonnation summafizlrtg apparatus (1 00) according to claim 6. wherein 

after deciding the start time of the partial video information based on ttie noise section, the decision device 
(107) sets the stop time of the partial video infonnation based on the tsne length of Itie noise setSlon concerned. 

8, The video infemtsiion summarizing ai:^afatus ( i oo) according to snym& of claims 4 to 7, whersiii 

the decision device (107) sets, within a preset time range, the time fength of the partiai video infonnation to 
be extracted. 

9, The video tnfomiation summanzing aj^aratus (100) according to any one of ctasms 1 to 8. wheresn 

the dectsion device HOT) sets the Importance of *e partiaf video information based on at least either tne 
7oe 0! the ttne ionqth of tie sound section used as reference to ths aecis on of the partiai video tnfoimafion to 

be SKtractsd, and 

ti-'ft gcr-jrdtu.n dov t o f 36 107) rMM',-* sj!iir)\i'y nf thu v'td<>o n o -mUar bv exttdctng the eattai vd^o 
n^o'Tiftt'or' on 11-'D tasfs of lis set irnpoitcnco of tt'c c ,^ 

10, TheviJc'O n o-n' j^icn tLti-naiu' iq.j;';''- _s ";g ' ^ ) v, ■■ 

dcci!^ >..'i 1< , '^7^--^-, , ', '-'o trdti'i' hdsod on tie siicnl feocton 

than that o: tiie partial video infonnation Dgsed on the noise section. 

11, The video sntortnarion summaii^inq aopsrsius {luui accordinq to ciaim 9 or "0, wneretn 

when the decided oiuraf pieces or paroa! video iniormation coincicJs v;-th one another, the oeasion device 
f107) merges she coincmnt pieces ot patta; viaeo mfontiaiion into a piece ct paitia! video irttormatton, and sets 
the iiTiDottance ot the merged eartiai video ;ntomiatiori basse on tne imDorranoe of each piece ot patltal vsdeo 
inrormation beinc metqsd at present, 

12 AviGODin'o!m.--nr sjmmrtfi/tnc} rr tjntodio! ox ractsnq ba&ed on aisd o info m 5' on jj'eo rfior.-^ptN.es o' partial 
Video mfortnatsof^ some parts of wdijo infomiatton ''rom the vtdeo infortr itior tc vvatc* tno ri. j o tffotmatton is 
added so that digest tnfonDation sumnrtart^ea m shorter time than the vidoo i nfo rr^a lor w 1 1 be gei let aicd from tne 
video information on the basis of the patfia! video infonnation extracted, said meth;jd compi ising- 
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c3 ctasfeificatton process of cirifc&ifvi'tij tho vidoo nformation Mo piuraf sound sections on the basis of the sound 
levels (n thejaodto information: 

a dec sfon process of cJscidipg ihs pattal wdco iriformation to be eiaracted on tte basis of at least either the 
lime-base position or me time iengfh of at least any one of the plural Sypes of sound sections cSassifisd in the 
s Video infofmijtion: and 

a gsnsratKjr} process of extracting tne dscicied partia! video information from the video infonrsation and gen- 
erate the digest mforrrjatiort. 

13 The viden n'orrr-' tor sunrnan^- nelhod according to claim 12, wherein 

10 i.t"e uf>(.i»)on piocess dec 3es at least either the starttime or the stop time of the partiaf video infofmation on 

*heod3ifeCfrf leas eithei ihetimp base oosit on orths time tsngth of atleast any oneofthepiuraS types of sound 

sec v^r^cabs tied i -ide^ info matron 

14 Tho viJcO n c n f\Qr tLii^itiJc i<i iieihod iccorctng to ctsiim 12 or 1S. whersin 

1S i^-firUs-^tfucilo^pftxi 1^ i-« 1'- H of tt;e sound k:-vo!sll-i(? vtd9r> ;nfot!rist!on intoai leastsound- 

0!^s• silent sixtnr? rtnd rc % > : ■ ;s ^ri?sf;; rartje of sound lev-rjls. 

15 f ho VI Jso ri onnattor summan/ ig Tielhod accorotng to any one of claims 12 to 1 4, 

iho decision process seis the fmporlance of the partial video information based on at ieast either the type or 
20 ihe Eine length of tn©soundse<Ston used as reference to the decision of the partial video Infomiatlon to be extracted, 
and 

the generation process makes a summary of the video information by extracting the partial video infotmation 
on the basis of the set importance of the partial video information. 

?5 16, A video infortrtation Siirnmari^ing progran embodied in a recording medsum which can be read by a cot^iputer in 
a video information stimmart^ing apparatus ( 1 00) for extracting; based on audio information, one or more pieces 
of partia! videa informafton as some parts of video informasion trom tl'te video information to wliich the audio infor- 
mation is added so that digest informiation summarized in shorter time than the video information will b& generated 
from the video infonriatton on the basts of the partial video Information extracted, the program causing the computer 

30 to function as: 

a classification device {1 03. 1 07) which classifies the video information into plural sound sections on the basis 
of the sound levels in the audio infonnation; 

a decision device (107) which decides the partial video information to be extracted en the basis of at least 
35 eitherthe time-base position or the time length of at (east any one of the plumi types of sound sections classified 

in the video information; and 

a generation device (108, 107) which extracts the decided partia! video Information from the video infofmation 
to generate the digest information. 

*5 17 Tho video n orrr^itior st.,mTian?-»go'-o9jamaceorofng to claim 16, wherein 

tj^s* d"o s I r cii^v ic«» (1 0 / jPC!do& at least either the start time or the stop time ot the partial video infonnation 
:)n thtj tMs ^ c'f at k>iii, f hei li6> Utn base position or the lime length of at Isi^t any one of the piunal types of 
souf^d scclons c<j«!i,ff -^d in hevJeo f'ormro" 

'ts 18 ThoviJt.j) non- j^icrtL-ri-narii' iqo JC' -c - '5 A-t. c - 

^!. cioSS(ftdl:w^€/r'' (10' '07 ^ t - xi thHnt s o' ■'iic sound Icvois the video infotmation into at 
SCI --J - i s s , . ^ - c cscEtargo of sound levels. 

19 1 1^ vde3 <^ nM'iLT !;L< n>^<d\i/ -\q Droqram accorarng lo clairii 16. wt^eresn 
5fJ 1 1 vleo !>ioi^ f^e^fice (107) sets the importance of me parttai video tnfomriation based on at least either the 

V X y tbt, tf-c ionqth of tie sound section used as reference to the decision of the partial video infonrtatJon to 

be extracted, and 

the qeneiatton device ^106. 107} ma!<es a summary of the video information by extracting the partia! video 
n-ormation on the basis of ine set importance of the parBal video infonr^ation. 



18 



EP 1 271 359 A2 






1S 



EP 1 271 359 A2 




20 



EP 1 271 359 A2 



HO 

us w 



21 



EP 1 271 359 A2 



d 

Ul 



OH 
O 
i~ Ui 



Mi — 
Oh- 



3fe 



2 0 

LU — 



C50 
— CL 

03 



22 



EP 1 271 359 ^ 



FIG.5 




DETECT! ON OF 
S ! LENT/^OSE 
SECTION 





SI 2 




READ Si LENT/HOSE 
SECTION 



[speci fy summary time [ --si 4 
decide start t!me amdhs^s 

iMPCRTANCE OF 
DiGEST SEGMENT 



NARROW DOm DiGEST SEGMENT |-S16 



SET M i N I MUM/MAX I MUM/ TYP 1 CAL h- S 1 7 
SEGMENT TIME 



S20 




DECIDE STOP TfME OF EACH 
DIGEST SEGMENT GENERATED 
UPON SiLENT SECTION 



DECIDE STOP T!M£ OF EACH 
DiGEST S£G^^ENT GENERATED 
UPON NOiSE SECTION 



SELECT DIGEST 
SEQUENT GENERATED 
!N ORDER OF IMPORTANCE 
BASED ON DIGEST TIME 



Q END J 



23 



EP 1 271 359 A2 



FIG. 6 




24 



EP 1 271 359 A2 



FIG. 7 
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